Using Prosodic Cues in Spoken Dialog Systems

In this paper we show how prosody can be used in spoken dialog systems. First, we describe the phenomena that prosodic analysis is concerned with and give examples why prosody is relevant in the context of spoken dialog processing. Then we examine prosody in the light of pattern classification. We show how prosodic events can be categorized. We detail how those prosodic events manifest themselves in the speech signal, i.e. what acoustic features are important. Having made clear what we want to distinguish and which features we use in order to do that, we present a statistical framework that enables us to reliably determine e.g. phrase boundaries and accents. After that we show how dialog systems can utilize prosodic information. The importance of prosody is strikingly demonstrated in the context of parsing word hypothesis graphs. In the VERBMOBIL speech–to–speech translation system, the use of boundary probabilities yields a speed–up of 92% and a 96% reduction of alternative readings. Segmentation and accentuation in the context of shallow linguistic analysis are other applications where prosody can be gainfully employed. For several new directions in prosody research (in the context of dialog systems) such as emotion detection, multi–lingual prosody, feature selection, and integration of prosodic knowledge in speech recognition, preliminary results are presented.